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Abstract 

This  report  describes  the  development  and  implementation  of  The  University  of 
Texas  Close  Call  Reporting  System,  an  anonymous  tool  used  to  collect 
information  about  potential  errors  that  do  not  reach  the  patient.  The  system  was 
originally  developed  through  experimentation,  using  a  fund  of  knowledge 
collected  in  the  aviation  industry.  A  multidisciplinary  health  care  team  adapted 
this  approach  to  develop  a  close  call  categorization  scheme  based  on  human- 
factor  principles.  This  system,  which  is  available  in  Web-based  and  paper 
formats,  allows  reporters  to  submit  close  calls  in  a  risk-free  manner  and  to  enable 
the  collection  of  rich  infonnation  about  the  etiology  of  close  calls.  This 
information  is  used  to  identify  areas  of  vulnerability  and  to  develop  interventions 
that  can  prevent  problems  from  recurring.  The  development  and  implementation 
processes  presented  provide  a  comprehensive  framework  that  can  be  used  for 
future  deployments  of  similar  patient  safety  systems. 


Introduction 

The  Institute  of  Medicine  (IOM)  reports  To  Err  Is  Human:  Building  a  Safer 
Health  System  and  Crossing  the  Quality  Chasm~  identified  several  ways  that 
health  care  professionals  might  increase  patient  safety.  One  method  discussed  in 
the  IOM  reports  and  other  recent  publications  is  a  reporting  system  that  focuses 
on  errors  and  close  calls.3  7  Asking  health  care  providers  to  report  errors  and  close 
calls  is  an  excellent  way  to  discover  events  not  revealed  by  other  methods  and  to 
capture  important  information  about  the  underlying  causes  of  these  events.1' s  10 
Furthermore,  reporting  systems  are  expected  to  identify  opportunities  for  process 
improvement  and  to  facilitate  the  development  of  innovative  interventions  that 
can  prevent  the  recurrence  of  these  events.11' 12  Given  the  importance  of  reporting 
systems  and  their  possible  benefits.  The  University  of  Texas  Close  Call  Reporting 
System  (UTCCRS)  was  developed. 

As  the  name  implies,  the  focus  of  the  UTCCRS  is  to  collect  information  about 
close  calls,  which  are  defined  as  potential  errors  that  are  caught  and  prevented 
before  reaching  the  patient  or  causing  hann  to  the  patient.  Close  calls  were  chosen 
as  the  focal  point  of  the  reporting  system  for  several  reasons.  First,  given  that 
close  calls  are  believed  to  have  underlying  causes  similar  to  those  for  errors,11' 12  a 
reporting  system  that  focuses  on  close  calls  should  allow  researchers  to  understand 
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the  etiology  of  both  close  calls  and  errors.  Second,  because  close  calls  are 
theorized  to  occur  much  more  frequently  than  errors,11-13  it  is  expected  that  many 
areas  in  need  of  process  improvement  would  be  identified.  Such  identification  can 
then  be  used  to  develop  and  test  the  efficacy  of  interventions  in  preventing  the 
recurrence  of  similar  close  calls  and  errors.13  Third,  while  many  health  care 
institutions  and  government  agencies  have  error  reporting  systems,  few  close  call 
reporting  systems  exist.4  One  reason,  therefore,  to  develop  a  system  focused 
exclusively  on  close  calls  is  to  determine  the  utility  such  a  system  might  have  for 
patient  safety.  Additionally,  focusing  on  close  calls  represents  a  paradigm  shift  for 
the  health  care  industry  in  which  health  care  is  moved  from  a  position  of 
reactivity  to  proactivity.  In  sum,  a  close  call  reporting  system  is  expected  to 
provide  health  care  researchers  and  professionals  with  information  about  the 
etiology  of  close  calls,  areas  for  improvement,  and  potential  interventions  for 
future  events,  thereby  increasing  patient  safety.  The  purpose  of  this  report  is  to 
detail  the  replicable  process  used  to  develop  and  implement  the  UTCCRS. 


Phase  one:  system  development 

The  initial  phase  of  the  development  of  the  UTCCRS  consisted  of  two 
complementary  activities:  (1)  fonning  an  advisory  board  that  would  provide  input 
and  guidance  about  the  development  of  the  UTCCRS;  and  (2)  reviewing  the 
research  literature  on  reporting  systems.  A  multidisciplinary  advisory  board  was 
appointed,  and  included  health  care  administrators,  nurses,  phannacists,  and 
physicians.  Given  that  the  UTCCRS  project  is  one  of  five  projects  conducted 
through  the  University  of  Texas  Center  of  Excellence  for  Patient  Safety  Research 
and  Practice  (whose  principal  investigator  is  Eric  J.  Thomas  of  The  University  of 
Texas-Houston  Medical  School),  funded  by  the  Agency  for  Healthcare  Research 
and  Quality  (AHRQ),  experts  from  the  Center  also  participated  on  the  advisory 
board.  These  experts  provided  valuable  information  about  translation  of  best 
practices  from  aviation  to  health  care,  human  factors,  organizational  learning,  and 
safety  attitudes  throughout  the  development  process. 

At  the  initial  advisory  board  meeting,  the  members  were  informed  about  the 
concept  of  a  close  call  reporting  system  and  reasons  that  such  a  system  would  be 
valuable  in  a  health  care  setting.  Additionally,  the  board  was  provided  with  an 
overview  of  the  systems  perspective  discussed  in  To  Err  Is  Human.  1  The  advisory 
board  was  asked  to  provide  expert  opinion  about  issues  identified  from  the 
literature13  that  are  important  to  consider  when  developing  a  reporting  system, 
including:  (1)  a  definition  of  a  close  call;  (2)  potential  barriers  that  might  be 
encountered  with  a  close  call  reporting  system  (i.e.,  whether  reporting  should  be 
anonymous  or  confidential);  (3)  possible  modalities  of  the  system  (i.e.,  paper, 
phone,  electronic);  (4)  feedback  mechanisms  to  reporters;  and  (5)  the  format  and 
content  of  the  system. 
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Definition  of  a  close  call 

Given  that  the  UTCCRS  focuses  on  close  calls,  it  was  important  that  the 
advisory  board  agree  on  a  close  call  definition.  Surprisingly,  many  members 
disagreed  about  what  constituted  a  close  call  versus  an  error,  with  definitions 
varying,  based  on  the  slippery  slope  of  “no  harm.”  The  advisory  board  adopted 
Bagian  and  Gosbee’s  definition  of  a  close  call,  an  “event  or  situation  that  could 
have  resulted  in  an  accident,  injury,  or  illness,  but  did  not,  either  by  chance  or 
through  timely  intervention.”3  The  adopted  close  call  definition  was  also  favored 
because  it  represented  a  conservative  conceptualization  of  close  calls,  was 
developed  by  recognized  leaders  in  patient  safety  initiatives,  and  was  expected  to 
make  reporters  feel  comfortable  reporting  potential  errors  as  opposed  to  actual 
errors. 

Potential  barriers  to  reporting 

The  advisory  board  discussed  reporter  anonymity — where  the  reporter  cannot 
be  linked  with  a  report,  versus  confidentiality — where  the  reporter-report  link  can 
be  established.  Consistent  with  research  literature  on  this  topic,1' 3’ 4' 14’ 15  the 
advisory  board  was  divided  about  whether  to  develop  an  anonymous  or  a 
confidential  system.  The  proponents  for  a  confidential  system  argued  that  it  was 
essential  to  followup  with  reporters  and  provide  them  with  feedback.  Advocates 
of  the  anonymous  system  noted  that  such  systems  would  be  perceived  as  safer  by 
reporters  and  would  reduce  the  fear  of  punitive  measures.  The  advisory  board 
identified  a  third  option — a  hybrid  system  that  was  confidential  until  the  report 
was  completed  and  reviewed,  at  which  time  the  report  would  be  de-identified  and 
entered  into  the  system  database.  While  advisory  board  members  noted  that  the 
decision  of  which  system  to  adopt  should  be  based  on  which  system  would 
maximize  reporting  of  close  calls,  there  was  no  reliable  research  found  on  the 
reporting  rates  of  the  three  options.  The  members  decided  unanimously  to  adopt 
an  anonymous  system  to  create  the  safest  environment  for  reporters  to  provide 
information  about  close  calls. 

System  modalities 

The  advisory  board  was  also  asked  to  detennine  possible  modalities  for  the 
UTCCRS.  Anecdotal  comments  from  the  advisory  board  led  to  the  belief  that 
different  professions  use  different  modalities.  For  example,  comments  revealed 
that  physicians  and  pharmacists  might  favor  electronic  and  telephone  reporting 
systems,  while  nurses  might  favor  paper  forms.  Advantages  to  a  Web-based 
system  included  the  ability  to  access  it  from  any  computer  and  that  a  short  amount 
of  time  is  needed  to  complete  such  a  report.  A  telephone  system  was  viewed  as  a 
plausible  solution  because  it  could  be  designed  similarly  to  transcription  services, 
which  are  already  used  by  health  care  providers.  Paper  fonns  had  the  advantage  of 
being  easy  to  store  in  health  care  institutions  that  lack  a  large  number  of  computer 
resources  and  are  the  most  familiar  report  format  for  many  providers.  The 
advisory  board  members  decided  that  the  feasibility  of  all  modalities  should  be 
examined.  After  examining  the  specifications  needed  to  get  all  three  modalities 
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working,  it  was  decided  to  eliminate  the  telephone  modality  because  of  concerns 
about  (1)  ease  of  use;  (2)  time  to  report;  and  (3)  assurance  of  anonymity.  It  was 
decided  to  adopt  both  the  electronic  and  paper  reporting  modalities  because  both 
met  these  specifications.  The  electronic  fonn  was  implemented  on  a  secure  Web 
site,  and  the  paper  form  consists  of  a  double-sided,  one-page  report. 

Feedback  to  reporters 

Researchers  mentioned  feedback  as  the  key  attribute  in  obtaining  multiple 
reports  from  providers.3  5  Not  surprisingly,  the  advisory  board  also  mentioned  the 
necessity  of  feedback  to  close  the  information  exchange  loop  with  the  reporter 
and  to  ensure  system  success.  However,  given  the  anonymous  nature  of  the 
system,  feedback  seemed  problematic.  To  address  this  issue,  a  method  of 
feedback  was  developed  for  both  the  individual  (i.e.,  reporter)  and  aggregate 
level.  At  the  individual  level,  after  a  reporter  enters  a  report  electronically,  he  or 
she  is  given  a  random  sequence  of  numbers  as  a  specific  identifier  for  that  report. 
The  reporter  can  then  visit  the  UTCCRS  Web  site  and  track  his  or  her  report  by 
entering  the  random  sequence  of  numbers.  The  tracking  feature  allows  researchers 
to  provide  specific  feedback  to  the  reporter  about  next  steps  that  would  result 
from  the  close  call  report,  such  as  any  best  practices  that  may  be  developed  to 
address  the  close  call.  The  feedback  mechanism  for  the  paper  form  works 
similarly,  with  preprinted  random  numbers  on  the  paper  form  that  can  be  tracked 
via  the  UTCCRS  Web  site.  At  the  aggregate  level,  feedback  sessions  for 
participating  institutions  were  discussed,  with  members  of  the  UTCCRS  team 
providing  frontline  providers  with  information  about  themes  and  trends  from  an 
institution’s  close  call  data. 

Format  and  content 

An  examination  of  the  literature  on  elements  common  to  existing  reporting 
systems  revealed  a  lack  of  consensus  concerning  whether  short  responses,  longer 
or  narrative  responses,  or  check  boxes  should  be  used  to  collect  information  about 
a  situation.  Researchers  in  favor  of  check  boxes  noted  that  check  boxes  might 
speed  up  the  time  it  takes  to  complete  a  report,  which  was  mentioned  as  a  possible 
corollary  to  increased  reporting.  Researchers  favoring  narration  argued  that  this 
response  format  produced  the  richest  type  of  data.  Interestingly,  no  research  was 
found  that  demonstrated  a  significant  difference  in  the  “richness”  of  the  data; 
rather,  the  noted  differences  were  anecdotal.  Given  the  lack  of  consensus  in  the 
literature  concerning  response  format,  the  advisory  board  recommended  a  hybrid 
approach  using  both  check  boxes  and  narrative  responses. 

With  respect  to  system  content,  the  advisory  board  members  noted  that  the 
primary  areas  of  interest  should  include  time  and  location  of  the  close  call; 
reporter  demographics;  description  of  the  close  call;  type(s)  of  close  call;  and 
reasons  for  close  call  occurrence.  The  advisory  board  recommended  the  fonnation 
of  a  multidisciplinary  work  team  to  detennine  specific  questions  for  these  areas  of 
interest.  A  work  team  was  formed,  consisting  of  practicing  health  care  providers, 
that  met  weekly  for  1  year. 
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Phase  two:  consensus  about  content 

The  work  team  conducted  an  indepth  examination  of  reporting  systems — 
including  some  site  visits  in  high-risk  industries  like  aviation,  petrochemicals,  the 
military,  and  health  care — to  better  inform  its  decisions  about  system  content. 

This  review  resulted  in  the  following  summaries: 

•  Aviation  Safety  Reporting  System  (ASRS).  The  ASRS  is 
administered  by  the  National  Aeronautics  and  Space  Administration 
and  offers  a  level  of  anonymity  (after  the  report  is  filed)  and 
confidentiality.  Because  the  ASRS  is  designed  for  reporting  all 
nonaccidents  (close  calls),  it  provides  an  opportunity  for  more 
prospective  review  and  preventive  intervention.  The  ASRS  receives 
upwards  of  30,000  reports  per  year.16 

•  Aviation  Safety  Action  Partnership  (ASAP),  This  new  system 
pennits  pilots  to  report  incidents  to  their  own  companies  with  the  same 
limited  immunity  provided  by  the  ASRS  national  program.  The  ASAP 
form  asks  the  reporter  to  categorize  the  incident  by  human-factors 
categories,  as  opposed  to  relying  only  on  an  event  narrative. 

•  Prevention  and  Recovery  Information  System  for  Monitoring  and 
Analysis.  This  system  collects  data  on  all  accidents  and  close  calls 
within  the  petrochemical  and  steel  industries.  Anonymity  and 
confidentiality  are  maintained. 

•  Nuclear  Regulatory  Commission  Allegations  System.  This  reporting 
system  is  maintained  by  the  Nuclear  Regulatory  Commission  and 
monitors  “all  safety  concerns”  within  the  nuclear  industry.  Anonymity 
is  not  guaranteed,  but  reporter  immunity  is  provided. 

•  Patient  Safety  Reporting  System  (PSRS).  This  reporting  system, 
used  by  the  Veterans  Health  Administration,  emphasizes  voluntary  and 
confidential  reports  of  nonintentionally  unsafe  close  calls  and  events 
within  a  nonpunitive  environment. 

Additionally,  to  aid  in  the  development  of  the  UTCCRS,  the  work  team  asked 
for  an  analysis  of  the  report  format  and  results  generated  from  the  current  incident 
report  system  at  The  University  of  Texas  M.  D.  Anderson  Cancer  Center.  The 
analysis  included  a  Pareto  chart  of  categories,  subcategories,  and  contributing 
factors  from  M.  D.  Anderson’s  incident  reports,  as  well  as  the  background  and 
demographic  questions  included  in  the  reports.  Such  analyses  infonned  decisions 
about  which  types  of  categories  (i.e.,  medication,  equipment  problems,  etc.)  and 
demographic  questions  to  include  in  the  UTCCRS;  the  work  team’s  decisions 
about  background  and  demographic  questions  and  close  call  categories  are 
presented  in  Table  1.  The  work  team  decided  to  include  free-text  narrative  fields, 
based  upon  the  ASAP  and  PSRS  models,  asking  the  following  questions:  (1) 
describe  the  event;  and  (2)  what  could  be  done  to  keep  it  from  recurring? 
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Table  1.  Primary  information  categories 

a.  Background  and  demographic  questions  for  the  UTCCRS 

Date  of  close  call 

1)  Time  of  close  call 

2)  Location  of  close  call 

3)  How  did  you  become  aware  of  the  close  call?  (Response  options:  involved,  witnessed, 
heard  about  it,  other) 

4)  Current  profession:  (Response  options:  dietician,  licensed/registered  nurse,  pharmacist, 
physical  therapist,  physician,  physician  assistant,  respiratory  therapist) 

5)  Total  years  worked  in  the  profession? 

6)  Total  years  worked  in  this  hospital? 

b.  Close  call  categories 

Blood/transfusion 
Diagnostic  test/procedure 
Equipment/devices 
Falls 

Medication 
Other  treatment 
Surgery 

Therapeutic  procedures 
Other 


The  work  team  struggled  to  identify  which  contributing  factors  to  use  to 
recognize  the  etiology  of  close  calls  with  respect  to  human  and  systems  factors. 
The  lack  of  consensus  about  contributing  factors  played  a  part  in  the  richest 
experience  of  the  present  study:  The  team  used  brainstorming  and  affinity 
problem-solving  methods  to  achieve  consensus.  The  initial  brainstonning  session 
resulted  in  8  broad  contributing  factors,  while  the  second  session — which 
occurred  1  month  after  the  first  session — resulted  in  1 8  specific  contributing 
factors.  However,  there  was  a  great  deal  of  confusion  about  the  specific  meaning 
of  the  factors. 

Given  the  confusion,  several  sentences  were  written  for  each  factor  that 
provided  detailed  information,  were  easy  to  understand  from  the  reporter’s 
perspective,  and  were  less  open  to  multiple  interpretations  because  of  their 
specificity.  The  contributing  factor  names  were  removed;  the  sentences  were 
written  on  sticky  notes  and  put  on  the  walls  of  a  conference  room,  and  the  work 
team  members  placed  the  sentences  from  the  contributing  factors  into  natural 
groups  (i.e.,  affinity).  The  affinity  exercise  resulted  in  17  groupings,  with  each 
having  between  4  and  9  sentences.  Names  were  generated  for  each  grouping  (e.g., 
contributing  factor);  Table  2  details  the  change  in  contributing  factors  across  the 
three  iterations.  An  Appendix  (available  at  the  Web  site  https://www.utccrs.org/ 
ccrs/Publications.html)  contains  contributing  factors  and  associated  drill-down 
sentences.  Once  the  work  team  reached  consensus  about  reporting  system  content, 
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a  technical  development  team  was  asked  to  translate  the  work  team’s  vision  of  the 
UTCCRS  into  a  fully  functioning  reporting  system. 


Table  2.  Contributing  factors  identified  as  the  result  of  the  brainstorming  and  affinity 
exercises 


First  brainstorming 
session 

Second  brainstorming 
session 

Affinity  exercise 

Communication 

Written  /  Verbal  Issues 

Communication 

Order  /  Transcription 

Order  or  T ranscription 

Issues 

Labeling  /  Packaging 

Labeling  /  Packaging 

Environmental 

Conditions 

Environmental  Factors 

Environmental  Factors 

Equipment/Technology  Issues 

Equipment/T  echnology 

Issues 

Information 

Information  Issues 

Research/Investigation 

Issues 

Personal  Issues 

Health  State  of  Healthcare 
Personnel 

Fatigue,  Sickness,  or  Stress 

Understanding  or  Forgetting 

Personnel  /  Workload 

Heavy  /  Complicated  Workload 

Workload 

Inadequate  Staffing 

Staffing  Issues 

Interpersonal  Issues 

Issues  Between  People 

Interruptions  /  Distractions 

Interruptions  /  Distractions 

Inadequate  Supervision 

Policy  /  Procedure 

Policy  /  Procedure 

Policy  Issues 

Clinical  Procedure  Issues 

Process  Issues 

Patient  Assessment 

Patient  /  Family  Issues 

Patient  /  Family  Compliance 

Training  /  Education  / 
Experience 

Competency 

Experience 

Educational  /  Training  Issues 

Training  Issues 

Slips  /  Lapses  in  Performance 

Phase  three:  UTCCRS  prototype 

The  ASAP  model  from  aviation  was  used  as  a  template  for  the  UTCCRS 
prototype,  which  was  developed  using  Microsoft™  Access.  While  developing  the 
prototype,  the  technical  development  team  commented  that  the  system  appeared 
to  be  visually  cluttered  because  of  the  number  of  background  and  demographic 
questions,  categories,  contributing  factors,  sentences  associated  with  the 
contributing  factors,  and  narratives.  This  issue  was  presented  to  the  work  team, 
who  decided  that  the  system  should  employ  a  series  of  drop-down  menus  that 
would  limit  the  amount  of  information  that  the  reporter  had  to  view  at  a  given 
time.  While  drop-down  menus  made  the  screen  much  less  cluttered,  the  approach 
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was  not  user-friendly  because  one  had  to  continually  scroll  up  and  down  the  drop¬ 
down  menus  to  select  all  of  the  appropriate  categories,  contributing  factors,  and 
contributing  factor  sentences.  The  work  team  was  consulted  again  and  asked  to 
propose  an  alternative  layout  for  the  system.  The  work  team  agreed  that  the 
reporting  system  should  be  divided  into  five  main  screens: 

1)  Background  and  demographic  questions 

2)  Narratives 

3)  Close  call  categories 

4)  Contributing  factors 

5)  Contributing  factors  sentences  based  on  conditional-logic  heuristics 

The  work  team  then  discussed  mechanisms  by  which  to  ensure  that  only  close 
calls  were  reported  in  the  system.  The  suggestion  that  was  implemented  was  to 
include  a  trap  question  at  the  beginning  of  each  report  to  ensure  that  only  close 
calls,  based  on  the  definition  adopted  by  the  advisory  board,  were  reported.  The 
question  asks  “Was  the  action  caught  before  it  was  carried  out  and  could  reach  the 
patient?”  A  “yes”  response  to  this  question  allows  the  reporter  to  continue  in  the 
system;  a  “no”  response  exits  the  reporter  out  of  the  report  system  and  informs 
them  to  use  an  incident  report.  Additionally,  the  narrative  section  contains 
specific  instructions  to  reporters  that  they  should  omit  any  information  that  might 
identify  patients  or  providers,  either  by  name  or  by  identification  number.  To 
further  ensure  de-identified  infonnation,  the  system  is  designed  so  that  reports  go 
initially  to  a  temporary  database  where  a  clinical  analyst  reviews  the  data  and 
removes  any  identifiers,  and  then  the  data  are  sent  to  the  final  database. 

The  technical  development  team  then  decided  it  was  necessary  to  field  test  the 
Microsoft  Access  prototype  with  frontline  providers  who  had  not  participated  on 
the  work  team.  Nurse,  pharmacist,  and  physician  volunteers  were  asked  to  test  the 
UTCCRS  prototype  and  enter  data  from  hypothetical  scenarios.  While  entering 
data,  these  providers  were  asked  for  their  ideas  and  concerns  about  system  design 
from  a  user  perspective.  Human-factors  analysis  using  standard  methodology  for 
computer  systems  was  also  conducted.  All  of  these  findings  led  to  the 
development  of  a  beta  model  using  Web-based  Java™  and  Oracle*  applications. 
This  set  of  applications  allowed  for  the  creation  of  several  enhancements  that 
responded  to  user  observations  and  the  human- factors  analysis.  Key 
enhancements  included — 

•  a  frequently-asked-questions  (FAQ)  section  in  the  electronic  version 
that  provides  information  about  aspects  of  the  reporting  system; 

•  a  help  section  in  the  electronic  version  that  contains  definitions  for  the 
primary  terms  relevant  to  the  reporting  system; 

•  report-editing  capability,  where  the  final  screen  of  the  report  displays 
all  of  the  reporter’s  responses  and  contains  hyperlinks  to  each  section 
of  the  report  that  the  reporter  can  use  to  change  his  or  her  responses; 
and 
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•  enhanced  security  options  in  the  electronic  version  to  eliminate  the 
possibility  of  hackers  overloading  the  system  with  automated  report 
writers. 

At  the  end  of  the  development  of  the  beta  model,  additional  volunteers  were 
approached  and  asked  to  enter  scenario  data.  Review  of  the  reporting  system  was 
conducted  again  using  standard  human-factors  methodology  for  computer 
systems.  Overall,  the  response  from  users  was  overwhelmingly  positive  with 
respect  to  ease  of  use,  design  of  the  system,  and  intuitiveness  of  the  system.  The 
work  team  and  advisory  board  reviewed  the  final  version  of  the  reporting  system 
and  provided  their  approval.  Once  the  final  electronic  version  of  the  UTCCRS 
was  approved,  it  was  deployed  to  a  secure  Web  site,  https://www.utccrs.org.  The 
paper  form  was  then  developed  to  reflect  the  same  questions  contained  in  the 
electronic  version  of  the  reporting  system.  Final  versions  of  the  reporting  system 
allowed  for  system  implementation.  The  following  sections  detail  the  UTCCRS 
implementation  process. 


Phase  four:  implementation 

Preimplementation 

Administrators  from  a  teaching  health  care  institution  in  Texas  discussed  the 
potential  usefulness  of  a  close  call  reporting  system  during  a  meeting  in  June 
2002.  This  institution  is  in  a  moderately  sized  city,  and  supports  the  surrounding 
rural  areas.  During  the  next  4  months,  the  UTCCRS  research  team  collaborated 
with  key  personnel  from  the  institution  to  develop  a  research  protocol  for  the 
reporting  system.  A  followup  meeting  occurred  in  October  2002,  which  resulted 
in  a  finalized  research  protocol  and  led  to  the  development  of  a  project  plan  for 
deployment  of  the  reporting  system.  This  plan  focused  on  discussing  the  reporting 
system  with  frontline  staff  and  managers,  and  identifying  and  addressing  barriers 
that  might  hinder  implementation  of  the  project. 

Presentations  were  made  to  a  representative  sample  of  frontline  staff  and 
managers  about  the  concept  and  purpose  of  the  UTCCRS,  expected  systems 
interventions  that  would  be  developed  based  on  the  reports,  and  the  expected 
impact  on  their  overall  patient  safety  efforts.  The  managers  identified  several 
concerns  about  the  reporting  system:  (1)  fictitious  reports,  (2)  reporting  of  errors 
instead  of  close  calls,  and  (3)  insignificant  reports.  Frontline  staff  mentioned 
concerns  about  (1)  anonymity,  (2)  punishment  by  a  supervisor  if  the  report  was 
recognized,  (3)  whether  the  Web-based  system  could  be  tampered  with,  and  (4) 
when  the  system  should  be  used.  All  issues  mentioned  by  managers  and  staff  were 
discussed  and  clarified  to  create  a  nonthreatening  environment  prior  to 
deployment  of  the  UTCCRS. 

System  launch 

The  reporting  system  was  launched  at  the  teaching  health  care  institution  in 
March  2003.  A  physician  from  the  participating  institution  provided  an  overview 
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of  the  UTCCRS  at  a  meeting  for  physician  staff,  while  the  UTCCRS  team 
provided  educational  sessions  for  other  health  care  professionals  at  all  shifts  over 
3  days.  Two  members  of  the  UTCCRS  team — a  clinical  nurse-specialist  and  an 
organizational  psychologist — conducted  the  educational  sessions.  In  addition  to 
using  standardized  materials  to  explain  how  and  when  to  use  the  reporting  system, 
the  educators  also  provided  a  context  for  the  importance  of  a  close  call  system  in 
today’s  health  care  system,  including  references  to  the  Institute  of  Medicine 
reports,  lessons  learned  from  the  aviation  industry,  and  the  similarity  of 
underlying  causes  for  close  calls  and  errors.  Additionally,  because  the  power  of 
storytelling  has  been  demonstrated  as  an  effective  way  to  educate  staff  about  new 
issues,17  anecdotal  stories  reflecting  nurses’  expertise  in  catching  close  calls  were 
shared  with  staff  during  the  educational  sessions  to  draw  parallels  between 
concepts  from  the  UTCCRS  and  examples  from  their  clinical  practice.  Providers 
were  informed  that  they  could  report  close  calls  via  a  paper  form  with 
accompanying  postage-paid  envelopes  or  electronically  from  any  computer 
connected  to  the  Internet  (by  accessing  the  Web  site  https://www.utccrs.org). 
Given  the  UTCCRS  project’s  goal  of  implementing  the  reporting  system  in 
multiple  institutions  and  the  need  for  reporter  anonymity,  institution  codes  were 
employed.  Institution  codes  are  a  string  of  letters  and  numbers  used  by  all 
reporters  at  an  institution.  The  codes  maintain  reporter  anonymity  while  providing 
the  UTCCRS  team  with  infonnation  that  allows  them  to  give  feedback  to  the 
correct  institution. 

Action  planning 

Electronically  submitted  reports  are  communicated  to  the  UTCCRS  team  in 
real-time  via  e-mail.  The  clinical  analyst  of  the  UTCCRS  team  screens  all  reports 
to  ensure  that  they  convey  an  event  that  is  consistent  with  the  definition  of  a  close 
call.  Reports  inconsistent  with  the  definition  are  communicated  back  immediately 
to  a  quality  liaison  from  the  participating  institution.  The  clinical  analyst 
communicates  any  urgent  close  calls  back  to  the  institution;  “urgent”  is  defined  as 
a  situation  where  immediate  action  is  needed  to  prevent  harm  to  a  patient  in  some 
part  of  the  institution.  For  example,  two  drugs  stored  in  the  wrong  bins  of  a 
medication-dispensing  machine  might  be  reported  through  the  system.  Because 
incorrect  storage  in  one  machine  might  mean  incorrect  storage  in  another  machine 
at  an  institution,  the  clinical  analyst  from  the  UTCCRS  team  would  inform  the 
quality  improvement/quality  assessment  (QI/QA)  liaison  of  that  institution  of  the 
problem.  It  is  possible  that  the  liaison  of  that  institution,  upon  examining  the  other 
medication-dispensing  machines  in  the  institution,  might  discover  the  same 
problem  in  multiple  machines.  The  clinical  analyst  also  de-identifies  any  reports 
that  inadvertently  contain  provider  or  patient  identifiers,  prior  to  entering  the 
information  into  the  final  database,  and  prepares  feedback  reports  for  the 
institutional  liaison  of  the  participating  institution. 

Feedback 

During  the  first  year  of  UTCCRS  implementation  at  the  participating 
institution,  feedback  sessions  were  developed  and  offered  on  a  quarterly  basis. 
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The  feedback  sessions  consisted  of  10-15  minute  presentations  highlighting  the 
content  of  the  reports  and  the  action  plans  developed  by  the  institution  to  address 
some  of  the  issues  mentioned  in  the  content.  Examples  of  these  action  plans  are 
failure  mode  and  effects  analysis;  newsletters  highlighting  medication  tips; 
education  sessions  from  the  phannacy  about  different  types  of  drugs;  and  online 
patient  safety  education. 


Preliminary  findings  about  UTCCRS  and  conclusion 

To  assess  the  utility  of  the  UTCCRS  in  terms  of  ease  of  use  and  impact  on 
patient  safety  efforts,  an  optional  anonymous  survey  was  added  to  the  electronic 
version  of  the  reporting  system.  Nineteen  providers  have  completed  the  survey  as 
of  March  2004,  and  their  responses  have  been  overwhelmingly  positive.  The 
survey  items  were  based  on  a  Likert  scale:  1  =  Strongly  Disagree,  2  =  Disagree, 

3  =  Neutral,  4  =  Agree,  and  5  =  Strongly  Agree.  The  items  and  their  respective 
mean  (M)  and  standard  deviation  (SD)  are  as  follows: 

•  I  believe  the  results  from  the  UTCCRS  will  be  used  to  develop 
interventions  that  will  make  health  care  safer  (M  =  4.21;  SD  =  0.79) 

•  I  am  satisfied  with  the  UTCCRS  (M  =  3.8;  SD  =  0.89) 

•  I  plan  to  recommend  the  UTCCRS  to  my  colleagues  (M  =  4.33; 

SD  =  0.66) 

•  The  UTCCRS  asks  confusing  questions  (M  =  2.0;  SD  =  0.86) 

•  I  believe  the  hospital  will  be  a  safer  place  by  having  the  UTCCRS 
(M  =  4.0;  SD  =  0.79) 

•  The  UTCCRS  is  easy  to  use  (M  =  4.25;  SD  =  0.97) 

•  I  plan  to  use  the  UTCCRS  again  to  report  a  close  call  (M  =  4.4; 

SD  =  0.68) 

In  sum,  the  UTCCRS  is  a  proactive  patient  safety  tool  that  allows  institutions 
to  learn  about  the  etiology  of  close  calls.  Reporters’  feedback  about  the  system 
has  demonstrated  that  the  system  is  easy  to  use  and  is  perceived  as  a  positive 
influence  on  patient  safety  efforts.  To  date,  the  UTCCRS  has  been  implemented 
in  six  institutions  using  the  implementation  and  maintenance  plan  described 
previously.  Currently,  the  project  team  is  fonnalizing  plans  to  share  themes  and 
lessons  learned  across  institutions  so  that  interorganizational  learning  and  a 
simultaneous  increase  in  patient  safety  may  occur. 
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